Fixation Prediction for 360° Video Streaming to Head-Mounted Displays

We study the problem of predicting the Field-of-Views (FoVs) of viewerswatching 360° videos using commodityHead-MountedDisplays (HMDs). Existing solutions either use the viewer’s current orientation to approximate the viewed tiles in the future, or extrapolate the FoVs using the historical orientations and dead-reckoning algorithms. In this paper, we develop €xation prediction networks that concurrently leverage sensorand content-related features to predict the viewer €xation in the future, which is quite di‚erent from the solutions in the literature. Œe sensor-related features include HMD orientations, while the content-related features include image saliencymaps andmotionmaps. We build a 360° video streaming testbed to HMDs, and recruit twenty-€ve viewers to watch ten 360° videos. We then train and validate two design alternatives of our proposed networks, which allows us to identify the beŠer-performing design with the optimal parameter seŠings. Trace-driven simulation results show the merits of our proposed €xation prediction networks compared to the existing solutions, including: (i) lower consumed bandwidth, (ii) shorter initial bu‚ering time, and (iii) short running time.